User Experiments with Search Services: Methodological Challenges for Measuring the Perceived Quality

For many people, search engines are a crucial entry point for their online activities today. People use search services to find relevant information of their interest on the web. Measuring the quality of search systems is a challenging task. User expectations have been proposed as one possible parameter for user satisfaction. The so-called confirmation/disconfirmation (C/D) paradigm, a widely used model to describe customer satisfaction, was used to predict the reaction of search engine users. Two studies were conducted to examine how prior expectations of users affect their perceptions of system quality of search engines. This paper describes the initial test designs, compares the methodical approaches and presents a concise summary of statistical results obtained using ANOVA.